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Abstract 

An interface device for use with a computer that provides locative data to a computer for tracking a user 
manipulatable physical object and provides feedback to the user through output forces. The physical object 
is movable in multiple degrees of freedom and is tracked by sensors for sensing the location and orientation 
of the object. A device processor can be responsive to the output of the sensors and can provide the host 
computer with information derived from the sensors. The host computer can provides images on a display, 
where the computer responds to the provided sensor information and force feedback is correlated with the 
displayed images via force feedback commands from the host computer. 
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Claims ^ - 





What is claimed: 

1. A device for use in conjunction with a computer display apparatus and a fixed surface, comprising: a 
stylus; means supportable on a fixed surface and coupled to said stylus for supporting said stylus while 
allowing at least a plurality of degrees of freedom in the motion of said stylus; and means for producing a 
stylus locative signal responsive to and corresponding with the position of the stylus at any point in time 
during its normal operation, said stylus locative signal providing information for use by a computer display 
apparatus. 

2. A device as recited in claim 1 wherein supporting means is a mechanical linkage. 

3. A device as recited in claim 2 wherein said stylus locative signal means is in communication with said 
mechanical linkage. 

4. A device as recited in claim 1 wherein said stylus locative signal means is in communication with said 
stylus. 

5. A device as recited in claim 1 further comprising: a remote unit having switch capable of being in an on 
state and an off state; and command means triggered by said switch when said switch is in its on state for 
generating a command signal for receipt by a computer. 

6. A device as recited in claim 5 wherein said remote unit is a foot pedal* unit. 

7. A device as recited in claim 2 wherein said mechanical linkage includes three individual components. 

8. A device as recited in claim 2 wherein said mechanical linkage includes at least three joints. 

9. A device as recited in claim 1 further comprising means for providing resistance to the motion of the 
stylus. 

10. A device as recited in claim 1 wherein said stylus has pencil-like configuration which can be manually 
manipulated. 

1 1. A device as recited in claim 1 further comprising: feedback means for generating force by said support 
means in response to force signals provided to said device, said force signals correlated to information 
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displayed on computer display apparatus. 

12. A method for interfacing a user and a computer display apparatus, comprising the steps of: providing a 
stylus; coupling to said stylus a support apparatus supportable on a fixed surface for supporting said stylus 
while allowing at least a plurality of degrees of freedom in the motion of said stylus; and providing means 
for producing a stylus locative signal responsive to and corresponding with the position of the stylus at any 
point in time during its normal operation, said stylus locative signal providing information for use by a 
computer display apparatus. 

13. A method as recited in claim 12 wherein support apparatus is a mechanical linkage. 

14. A method as recited in claim 13 wherein said stylus locative signal means is in communication with 
said mechanical linkage. 

15. A method as recited in claim 12 wherein said stylus locative signal means is in communication with 
said stylus. 

16. A method as recited in claim 12 further comprising the steps of: providing a remote unit having switch 
capable of being in an on state and an off state; and providing a command signal generator triggered by 
said switch when said switch is in its on state for generating a command signal for receipt by a computer. 

17. A method as recited in claim 16 wherein said remote unit is a foot pedal unit. 

18. A method as recited in claim 13 wherein said mechanical linkage includes three individual 
components. 

19. A method as recited in claim 13 wherein said mechanical linkage includes at least three joints. 

20. A method as recited in claim 12 further comprising means for providing resistance to the motion of the 
stylus. 

21. A method as recited in claim 12 wherein said stylus has pencil-like configuration which can be 
manually manipulated. 

22. A device as recited in claim 1 1 further comprising the steps of: providing feedback means for 
generating force by said support means in response to force signals provided to said device, said force 
signals correlated to information displayed on computer display apparatus. 

23. In a computer system having a main unit and cursor control functions and command control functions 
which are triggered remotely from said main unit, a device for interfacing a user and computer, comprising: 
a remote command control unit separate from a cursor control unit having a switch capable of being in an 
on state and an off state; command means triggered by said switch when said switch is in its on state for 
generating a command signal representative of a command from a user to a computer; and transmission 
means for transmitting said command signal to said computer. 

24. A device as recited in claim 21 wherein said remote command control unit is a foot pedal unit. 

25. In a computer system having a main unit and cursor control functions and command control functions 
which are triggered remotely from said main unit, a system for interfacing a user and computer comprising 
the steps of: providing a remote command control unit separate from a cursor control unit having a switch 
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capable of being in an on state and an off state; generating a command signal representative of a command 
from a user to a computer when said switch is in its on state; and transmitting said command signal to said 
computer. 

26. A system as recited in claim 23 wherein said remote command control unit is a foot pedal unit. 



Description 



FIELD OF THE INVENTION 

[0001] The present invention relates to a computer-human interface device, and more particularly it relates 
to a stylus coupled to a supportable mechanical linkage for providing commands to a computer. 

* BACKGROUND OF THE INVENTION 

[0002] As the use of Computer Aided Design (CAD) Systems becomes more widespread, the need for 
cursor and command control devices which accurately and easily track three-dimensional position or 
motion is also growing. Devices which allow users to control a cursor with three-dimensional position 
and/or orientation commands are available for various applications. Among them are many hand-held input 
devices which allow users to interact with a host processor by controlling the position of a cursor or 
manipulating graphic objects on a computer screen. While these devices allow three-dimensional 
information to be transmitted to a computer they do not allow the user to use gestures and motions which 
are natural to the user. 

[0003] For example, a prior art device of the type which is used for three-dimensional control involves the 
use of accelerometers to transduce the position and orientation of a stylus in space as described in U.S. Pat. 
No. 4,839,838. This device makes no provisions so the stylus can be grasped in a manner which makes use 
of finger dexterity nor does it include mechanical support to reduce fatigue or enhance user control or 
dexterity. 

[0004] Another prior art example is an ultrasonic position-locating device like the one shown in U.S. Pat. 
No. 5,142,506. This device transduces position and orientation by triangulating ultrasonic signals. As with 
the prior art previously described, this device uses a free- floating stylus which includes no provisions for 
mechanical support to reduce fatigue or enhance user control or dexterity. Furthermore, this device is used 
with a stylus that is grasped in the palm of the hand. The use of such a stylus precludes fine positioning 
with the fingers and greatly reduces the dexterity of the user to manipulate position and orientation. In 
addition, this device is used with digital buttons on the stylus to send to the computer command signals. A 
button of this type is commonly called a "clicker" on a "mouse." Because such buttons are mechanically 
coupled to the free-floating stylus, it is difficult to push the buttons while maintaining the position and 
orientation of the stylus. By pushing down on the button, the user will necessarily move the stylus from its 
desired position. Accordingly, these commands are difficult to control under many circumstances. 

SUMMARY OF THE INVENTION 

[0005] In the present invention, the user holds a stylus which is supported by a support apparatus on a fixed 
surface so that the user can easily manipulate the stylus in free space to interact with a computer. The 
three-dimensional motion of the user is translated through the stylus and mechanical linkage to a processor 
which communicates with the computer, thus allowing commands to be sent to the computer which track 
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the three-dimensional motion of the user. Therefore, cursor control in three-dimensions on the 
two-dimensional computer screen is possible. 

[0006] In one embodiment, the stylus is supportable on a fixed surface by a set of mechanical linkages 
which include individual components joined together by a sufficient number of joints to allow several 
degrees of freedom in the motion of the stylus. These mechanical linkages provide mechanical leverage, 
friction, counter-weighing, and/or spring resistance in order to reduce fatigue of the user and to provide 
support to enhance the stability and dexterity of user manipulation of the stylus, 

[0007] In the aforementioned embodiment of the present invention, the joints of the mechanical linkages 
are coupled to sensors which provide information about their position. Such information is transmitted to a 
microprocessor so that position and orientation of the stylus can be computed using kinematic equations 
associated with or related to the particular linkage system. In another embodiment, position and orientation 
of the stylus is sensed through the use of ultrasonic, magnetic, or optical position and orientation sensors 
mounted on the stylus. 

[0008] Another aspect of the present invention includes a remote control unit which is used in place of a 
command clicker on the stylus. For example, a foot pedal or hand-held unit for the user's opposite hand is 
included to provide command control to the computer. Accordingly, manual dexterity of stylus 
manipulation is not compromised. 

DESCRIPTION OF THE DRAWINGS 

[0009] FIG. 1 is a perspective view of an embodiment of the present invention; 

[0010] FIGS. 2 A and 2B are block diagrams over- viewing two different electronic hardware configurations 
of the present invention; 

[001 1] FIG. 3 is a flow chart describing the main software loops for two different electronic hardware 
configurations shown in FIG. 2; 

[0012] FIGS. 4 A and 4B are flow charts describing two different interrupt service routines for serial output 
to host computer; 

[0013] FIG. 5 is a perspective representation of another embodiment of the present invention; 
[0014] FIG. 6 is a perspective view of still another embodiment of the present invention; 
[0015] FIG. 7 is a perspective representation of another embodiment; 
[0016] FIG. 8 is a perspective view of another embodiment; 

[0017] FIG. 9 shows an embodiment of the resistance mechanism of the present invention; 
[0018] FIG. 10 shows another embodiment of the resistance mechanism; and 
[0019] FIG. 1 1 shows yet another embodiment of the resistance mechanism. 
DESCRIPTION OF PREFERRED EMBODIMENTS 
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[0020] Referring to FIG. 1, a stylus 1 1 is shown attached to a support apparatus which is, in turn, supported 
on a fixed surface. By electrical and electronic configurations described below, the stylus 1 1 is adapted to 
provide data from which a computer or other computing means such as a microprocessor can ascertain the 
position and orientation of the stylus as it moves in three-dimensional space. This information is then 
translated to an image on a computer display apparatus. The stylus 1 1 may be used, for example, by an 
operator to change the position of a cursor on a computer controlled display screen by changing the 
position and/or orientation of the stylus, the computer being programmed to change the position of the 
cursor in proportion to the change in position and/or orientation of the stylus. In other words, the stylus 1 1 
is moved through space by the user to designate to the computer how or where to move the cursor on a 
computer display apparatus. 

[0021] the stylus 1 1 is a pen-like stick which can be manipulated between the fingers, allowing for much 
better control and fine dexterity as compared to full hand grips or palm-supported styluses used by some 
prior art inventions. While the stylus 1 1 is described in terms of manual manipulation, other stylus 
configurations are envisioned by the present invention, in particular, this invention includes manipulation 
by those unable to manually manipulate a pen. A stylus of the present invention, need not be linear, but 
may be curved or angled so that it may be held, for example, by the foot or the mouth of a person. 

[0022] Because the stylus is supported by a support apparatus which is in turn supported by a fixed surface 
or other stabilizing configuration, the user can manipulate the stylus with a minimum of effort. Also, if the 
user chooses to discontinue using the stylus, it is capable of maintaining its position in space, unattended. 
While FIG. 1 shows that preferred embodiment of the present invention, FIGS. 5-8 show alternative 
embodiments, such which are also contemplated under the present invention. It is preferable that the stylus 
have enough degrees of freedom to enable it to move through the mechanical linkage to give the user the 
amount of flexibility needed to move the cursor as desired. In FIG. 1, six degrees of freedom are shown 
and are labeled as Axis' 16. This, of course, provides maximum flexibility. Fewer degrees of freedom, such 
as a plurality of degrees of freedom, may also be sufficient depending on the application. 

[0023] In one embodiment, the stylus is connected to rigid individual components which are joined 
together by joints. While not shown, other types of support apparatus 1 are included in the present invention. 
For example, other configurations include a semi-flexible rod or any other moveable while supportive 
configuration which can support the stylus in the manner described herein. 

[0024] In FIG. 1, a mechanical linkage pursuant to the present invention is depicted. The stylus 1 1 is 
coupled to supportable mechanical linkages via joint 12 which, in the shown embodiment, houses sensors 
13A and 13B. Linkage 14, is connected, via joint 15 having position sensors 16A and 16B, to linkage 17. 
Joint 18 in turn connects linkage 17 with the vertical base protrusion 20 which emanates from the base 21. 
The sensors are used to produce a stylus locative signal which is responsive to and corresponds with the 
position of the stylus at any point in time during its normal operation. The stylus locative signal is used to 
provide information for use by a computer display apparatus of a computer. The term "joint" as used herein 
is intended to mean the connection mechanism between individual linkage components. In fact, two 
separate moveable members can be joined; such together forming a joint. 

[0025] The base 21, if necessarily, can be immobilized by securing it onto the fixed surface 23 by way of 
bolt, screw or other attachment mechanism 22. Moreover, the present invention implements mechanical 
leverage and rubbing friction (not shown) between the supportable mechanical linkages 14 and 17 and the 
joints 12, 15 and 18 in order to provide resistance and support so as to allow better dexterity than can be 
achieved with free-floating stylus trackers. This support and leverage aids in reducing the fatigue 
associated with manipulating the free-floating stylus 11. 
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[0026] As mentioned above, attached to each joint 12, 15 and 18 are sensors 13A, 13B, 16A, 16B, 19A, 
and 19B, respectively. These sensors sense the angle differential before and after motion of the two 
segments connected by that joint. The sensors can be, for example, optical incremental encoders, optical 
absolute encoders and potentiometers. Because the three-dimensional position and/or orientation tracking 
is achieved mechanically, this preferred embodiment avoids problems that magnetic and ultrasonic sensors, 
such as those shown in the prior art, encounter with metal and shadowing. However, as shown in FIG. 1, if 
desired, sensing means can be used to track the position and/or orientation of the stylus by mounting a 
single or several orientation sensors in the stylus 1 1 itself, such referred to as a stylus mounted sensor 11. 
An ultrasound, magnetic, optical or position and orientation sensor can be used as the stylus mounted 
sensor 1 1 . 

[0027] FIG. 1 also shows a clicker button 24 on stylus 11. The button is connected to a switch which when 
in the on state, sends a signal to the computer giving it a command. In order to provide for accuracy when 
sending commands, this invention also includes a remote clicker unit. Therefore, since the clicking motion 
occurs at a distant location from the cursor control, there is little or no opportunity to accidently move the 
cursor while making a command. FIG. 1 shows two configurations for implementing this aspect of the 
present invention. The first is identified as an alternate hand-clicker 25, the second as foot pedal 26. 

[0028] Digital buttons 27 and 28 which are connected to switches (not shown) on the remote attached 
peripherals such as a hand-held clicker unit 25 or a foot pedal 26, respectively, can generate additional 
digital input such transmitted through lines 25' and 26' respectively. Either of the shown ancillary remote 
command units, such including the hand unit 25 and the foot pedal 26 configurations, are favorable 
methods of inputting digital commands by command hardware or software (not shown) because pressing 
the button 27 or 28 does not compromise a user's ability to hold the stylus steady whereas pressing any 
button 24 on the stylus does compromise stylus stability. 

[0029] Referring to FIG. 2 A, the sensors 13 A, 13B, 16 A, 16B, 19A and 19B, along with any peripherals 
24, 25 or 26, can send their digital signals directly to a versatile floating-point processor or microprocessor 
32A which is controlled by software stored in a digital ROM (Read-Only Memory) via transmission line 
32' or another form of transmission, i.e., radio signals. As shown in FIG. 2B, an alternative embodiment 
can be used to lessen the demands on the floating-point processor or microprocessor 32B. The digital 
inputs of the sensors 13 A, 13B, 16A, 16B, 19A and 19B can be sent indirectly to the floating-point 
processor or microprocessor 32B by way of dedicated chips 13C, 13D, 16C, 16D, 19C and 19D, which 
pre-process the angle sensors' signals before sending them via bus 3 1 to the floating-point processor or 
microprocessor 32B which would combine these signals with those from the peripherals 24, 25 or 26. An 
8-bit data bus plus chip-enable lines allow any of the angle determining chips to communicate with the 
microprocessor. Moreover, reporting the status of peripherals 24, 25 or 26 includes reading the appropriate 
digital switch and placing its status in the output sequence array. Some examples of specific electronic 
hardware usable for sensor pre-processing include quadrature counters, which are common dedicated chips 
that continually read the output of an optical incremental encoder and determine an angle from it, Gray 
decoders, filters, and ROM look-up tables. 

[0030] The single-chip configuration of FIG. 2 A is most applicable where the angle sensors 13 A, 13B, 
16A, 16B, 19A and 19B are absolute sensors, which have output signals directly indicating the angles 
without any further processing, thereby requiring less computation for the microprocessor 32A and thus 
little if any pre-processing. The multi-chip configuration of FIG. 2B is most applicable if the sensors 13 A, 
13B, 16 A, 16B, 19A and 19B are relative sensors, which indicate only the change in an angle and which 
require further processing for complete determination of the angle. 

[0031] In either configuration, if the microprocessor 32 A or 32B is fast enough, it will compute stylus 1 1 
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position and/or orientation (or motion, if desired) on board the embodiment and send this final data through 
any standard communications interface such as an RS-232 serial interface 33 on to the host computer 
system 34 and to computer display apparatus 34" through transmission line 34' or another form of 
transmission. If the microprocessor 32A or 32B is not fast enough, then the angles will be sent to the host 
computer 34 which will perform these calculations on its own. 

[0032] In addition to the single-chip and multi-chip configurations, a variation may consist of a single 
microprocessor which reads the peripherals, obtains the angles, possibly computes coordinates and 
orientation of the stylus 11, and supervises communication with the host computer 34. Another variation 
may consist of dedicated sub-circuits and specialized or off-the-shelf chips which reads the peripherals, 
monitors the angle sensors 13 A, 13B, 16A, 16B, 19A and 19B, determine the joint angles, and handle 
communications with the host computer 34, all without software or a microprocessor 32A or 32B. 

[0033] Software is only included in the two microprocessor-based configurations shown in FIGS. 2A and 
2B. The more dedicated hardware a given configuration includes, the less software it requires. The 
software consists of a main loop (FIG. 3) and an output interrupt (FIGS. 4A and 4B). 

[0034] Referring to FIG. 3, the main command loop responds to the host computer 34 and runs repeatedly 
in an endless cycle. With each cycle, incoming commands from the host computer are monitored 36 and 
decoded 37, and the corresponding command subroutines for reporting angles, thus stylus position and/or 
orientation (see FIGS. 4A and 4B), are then executed 38. Two possible subroutines are shown in FIGS. 4A 
and 4B. When a subroutine terminates, the main command loop resumes 39. Available command will 
include but are not limited to: reporting the value of any single angle, reporting the angles of all six angles 
at one time, reporting the values of all six angles repeatedly until a command is given to cease 
aforementioned repeated reporting, reporting the status of peripheral buttons, and setting communications 
parameters. If the angle sensors require preprocessing, these commands will also include resetting the angle 
value of any single angle or otherwise modifying preprocessing parameters in other applicable ways. 
Resetting pre-processed angle values or preprocessing parameters does not require output data from the 
device. The microprocessor 32A or 32B simply sends appropriate control signals to the preprocessing 
hardware 13C, 13D, 16C, 16D, 19C, and 19D. If the microprocessor or floating-point processor is fast 
enough to computer stylus coordinates and orientation, these commands will also include reporting the 
stylus coordinates once reporting the stylus coordinates repeatedly until a command is given to cease, 
ceasing aforementioned repeated reporting, reporting the stylus coordinates and orientation once, reporting 
the stylus coordinates and orientation repeatedly until a command is given to cease, ceasing 
aforementioned repeated reporting. If force reflection is supported, these commands will also include 
reporting the forces felt by any single joint, setting the resistance of any single joint, and locking or 
unlocking a joint. 

[0035] Any report by the subroutines of FIGS. 4A and 4B of a single angle value requires determining 41 
the given joint angle. For the single-chip configuration shown in FIG. 2 A, this subroutine directly reads the 
appropriate angle sensor 42 from among sensors 13 A, 13B, 16 A, 16B, 19 A, and 19B. For the multi-chip 
configuration shown in FIG. 2B, this subroutine reads the outputs 43 of pre-processing hardware 13C, 13D, 
16C, 16D, 19C, and 19D which have already determined the joint angles from the outputs of the sensors 
13A, 13B, 16A, 16B, 19A, and 19B. Any report of multiple angles is accomplished by repeatedly executing 
the subroutine for reporting a single angle. The subroutine is executed once per angle, and the values of all 
angles are then included in the output sequence array. If the optional parts of the subroutines 45 are 
included, then these subroutines become the coordinate reporting subroutines. Many other commend 
subroutines exist and are simpler yet in their high-level structure. 

[0036] After determining the given joint angle, the microprocessor 32A or 32B creates an output sequence 
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44A or 44B by assembling an array in a designated area of processor memory which will be output by the 
microprocessor's communications system at a given regular communications rate. The sequence will 
contain enough information for the host computer 34 to deduce which command is being responded to, as 
well as the actual angle value that was requested. Returning to FIG. 3, a query 36 in the main command 
loop asks whether the previous command requested repeated reports. If so, the main command loop is 
initiated accordingly. The communications output process (not shown) may be as simple as storing the 
output data in a designated output buffer, or it may involve a standard set of communications interrupts that 
are an additional part of the software. Setting communications parameters does not require output data 
from the device. The microprocessor 32 A or 32B simply resets some of its own internal registers or sends 
control signals to its communications sub-unit. 

[0037] To report the stylus 1 1 1 coordinates, three of the five or six angle values pre-read and knowledge of 
link lengths and device kinematics are incorporated to compute stylus 1 1 coordinates. These coordinates 
are then assembled in the output sequence array. 

[0038] To report the stylus' 1 1 orientation, at least five angle values are read and knowledge of link lengths 
and device kinematics are incorporated to computer stylus 1 1 orientation. The orientation consists of three 
angles (not necessarily identical to any joint angles) which are included in the output sequence array. 

[0039] Forces felt by a joint, setting a joint's resistance, and locking or unlocking a joint are reported by 
using interaction of the microprocessor 32A or 32B with forced-reflecting hardware (not shown). 
Reporting forces felt by a joint uses a force sensor mounted on the joint and then places the resulting value 
in the output sequence array. To set a joint's resistance and locking or unlocking a joint control signals 
reading from a force sensor to force-reflection hardware but do not require any output data of the device. 

[0040] Also contemplated in the present invention is computer software and hardware which will provide 
feedback information from the computer to the stylus (not shown). This type of implementation is known 
in robotics and thus is easily incorporated into a system including the present invention. When a surface is 
generated on the computer screen, the computer will send feedback signals to the mechanical linkage 
which has force generators for generating force in response to the cursor position on the surface depicted 
on the computer screen. Force is applied for example, by added tension in the joints which is in proportion 
to the force being applied by the user and in conjunction with the image on the screen. 

[0041] The various configurations of the mechanical linkages shown in FIG. 5, FIG. 6, FIG. 7 and FIG. 8 
which have different numbers of individual components and joints than shown in FIG. 1 are illustrative of 
the numerous possible configurations which can provide varying degrees of freedom inherent in the present 
invention. Referring to FIG. 5, FIG. 6 and FIG. 8, note that a rounded object such as a ball can act as a joint 
having motion in three degrees of freedom. In conjunction with other mechanical linkages and attachments, 
this permits sufficient degrees of freedom for the purposes of the present invention. In each figure, the 
orientation of the degrees of freedom of each joint is depicted by curved lines, numbered consecutively. 

[0042] Briefly, FIG. 5 shows a rounded joint 46 at the base such that three degrees of motion are available 
at that joint. FIG. 6 shows a three-dimensionally rotatable rounded joint 47 at the base through which one 
mechanical linkage can slide linearly and where the base is attached to a fixed surface 48 such that the 
surface does not prohibitively impede the movement of the device. FIG. 7 shows an embodiment where the 
basal connection can slide about the base in a two-dimensional plane in the cross configuration 49 on base 
5 1 . FIG. 8 shows a three-dimensionally rotatable rounded joint 52 at a perpendicular projection from the 
base 53 through which one mechanical linkage 54 can slide linearly through the joint 52. 

[0043] While any of the above discussed configurations or others can be used in accordance with the 
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present invention, FIGS. 9-11 show different mechanisms for providing resistance to the manual 
manipulation of the stylus by the user. FIG. 9, for example, shows return or tension springs 56 on each 
joint of the embodiment shown in FIG. 1. In an alternative embodiment, FIG. 10, shows counter- weights 
57 on each joint. Moreover, FIG. 11, shows a combination of a return or tension spring 56, a counter-wight 
57 and a compression spring 58. The arrangement of the resistance mechanism used should depend upon 
the configuration stylus mechanical linkage combination, such arrangement preferably chosen to maximize 
the ease with which the user can manipulate the stylus 1 1 in free space in accordance with the present 
invention. 

***** 
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